Making Good on LSTMs' Unfulfilled Promise

LSTMs promise much to financial time-series analysis, temporal and cross-sectional inference, but we find that they do not deliver in a real-world financial management task. We examine an alternative called Continual Learning (CL), a memory-augmented approach, which can provide transparent explanations, i.e. which memory did what and when. This work has implications for many financial applications including credit, time-varying fairness in decision making and more. We make three important new observations. Firstly, as well as being more explainable, time-series CL approaches outperform LSTMs as well as a simple sliding window learner using feed-forward neural networks (FFNN). Secondly, we show that CL based on a sliding window learner (FFNN) is more effective than CL based on a sequential learner (LSTM). Thirdly, we examine how real-world, time-series noise impacts several similarity approaches used in CL memory addressing. We provide these insights using an approach called Continual Learning Augmentation (CLA) tested on a complex real-world problem, emerging market equities investment decision making. CLA provides a test-bed as it can be based on different types of time-series learners, allowing testing of LSTM and FFNN learners side by side. CLA is also used to test several distance approaches used in a memory recall-gate: Euclidean distance (ED), dynamic time warping (DTW), auto-encoders (AE) and a novel hybrid approach, warp-AE. We find that ED under-performs DTW and AE but warp-AE shows the best overall performance in a real-world financial task

A temporal continual learning framework for investment decisions

Temporal continual learning (TCL) is introduced in this thesis as an extension of continual learning (CL). While traditional CL has been applied to sequential tasks, extending CL to TCL aims to allow machines to accumulate specific knowledge of temporal states, to address concept drift (CD) problems. This approach is shown to hold considerable benefits in domains where non-stationary time-series are used for decision-making, particularly in finance. A TCL framework called continual learning augmentation (CLA) is introduced, to drive long-term decision making in complex, multivariate, temporal problems. Moreover, CLA uses an external memory structure to store learner parameters from particular past temporal states for recall in the future. The contributions of this work are fourfold: First, a temporal, state-based, external memory structure is developed. Second, this is used to memory augment well-understood base-learners, such as LSTM, feed-forward neural networks (FFNN) and linear regression. Third, a remember-gate, based on residual change, learns in an open-world fashion to define different states for which learner-parameters are stored along with a contextual reference of the state. Fourthly, a memory recall gate is developed, based on various time-series similarity approaches, which can compare the current input space with the contextual references stored in memory, recalling the most appropriate learner parameters for use in the current period. In testing, CLA is found to improve the performance of LSTM, FFNN, and linear regression learners applied to a complex, real-world finance task: stock selection in international and emerging equities investing. Several different similarity approaches are tested in CLA's remember-gate, with dynamic time warping (DTW) outperforming simple Euclidean distance (ED), while auto encoder (AE) distance is found to both mitigate the resource overheads of DTW and provide better performance. A hybrid approach is also introduced, warp-AE, which performs well. In addition, a visualisation is introduced to allow CLA to be interpreted by domain experts in terms of which memory did what and when. A complex application is used to test TCL and a five-point statistical testing framework is introduced. This thesis elucidates the research of the last five years regarding TCL. Keywords: Continual learning, time-series, memory, neural network

Efficacy of customised foot orthoses in the treatment of achilles tendinopathy : study protocol for a randomised trial

BACKGROUND: Achilles tendinopathy is a common condition that can cause marked pain and disability. Numerous non-surgical treatments have been proposed for the treatment of this condition, but many of these treatments have a poor or non-existent evidence base. The exception to this is eccentric calf muscle exercises, which have become a standard non-surgical intervention for Achilles tendinopathy. Foot orthoses have also been advocated as a treatment for Achilles tendinopathy, but the long-term efficacy of foot orthoses for this condition is unknown. This manuscript describes the design of a randomised trial to evaluate the efficacy of customised foot orthoses to reduce pain and improve function in people with Achilles tendinopathy. METHODS: One hundred and forty community-dwelling men and women aged 18 to 55 years with Achilles tendinopathy (who satisfy inclusion and exclusion criteria) will be recruited. Participants will be randomised, using a computer-generated random number sequence, to either a control group (sham foot orthoses made from compressible ethylene vinyl acetate foam) or an experimental group (customised foot orthoses made from semi-rigid polypropylene). Both groups will be prescribed a calf muscle eccentric exercise program, however, the primary difference between the groups will be that the experimental group receive customised foot orthoses, while the control group receive sham foot orthoses. The participants will be instructed to perform eccentric exercises 2 times per day, 7 days per week, for 12 weeks. The primary outcome measure will be the total score of the Victorian Institute of Sport Assessment - Achilles (VISA-A) questionnaire. The secondary outcome measures will be participant perception of treatment effect, comfort of the foot orthoses, use of co-interventions, frequency and severity of adverse events, level of physical activity and health-related quality of life (assessed using the Short-Form-36 questionnaire - Version two). Data will be collected at baseline, then at 1, 3, 6 and 12 months. Data will be analysed using the intention to treat principle. DISCUSSION: This study is the first randomised trial to evaluate the long-term efficacy of customised foot orthoses for the treatment of Achilles tendinopathy. The study has been pragmatically designed to ensure that the study findings are generalisable to clinical practice. TRIAL REGISTRATION: Australian New Zealand Clinical Trials Registry Number: ACTRN12609000829213

Foundations of programming, statistics, and machine learning for business analytics

Business Analysts and Data Scientists are in huge demand, as global companies seek to digitally transform themselves and leverage their data resources to realize competitive advantage. This book covers all the fundamentals, from statistics to programming to business applications, to equip you with the solid foundational knowledge needed to progress in business analytics. Assuming no prior knowledge of programming or statistics, this book takes a simple step-by-step approach which makes potentially intimidating topics easy to understand, by keeping Maths to a minimum and including examples of business analytics in practice. Key features:· Introduces programming fundamentals using R and Python· Covers data structures, data management and manipulation and data visualization· Includes interactive coding notebooks so that you can build up your programming skills progressively Suitable as an essential text for undergraduate and postgraduate students studying Business Analytics or as pre-reading for students studying Data Science

Measurement of the three-dimensional distribution of radiation dose in grid therapy

A single large dose of megavoltage x-rays delivered through a grid is currently being utilized by some centres for palliative radiotherapy treatments of large tumours. In this note, we investigate the dosimetry of grid therapy using two-dimensional film dosimetry and three-dimensional gel dosimetry. It is shown that the radiation dose is attenuated more rapidly with depth in a grid field than an open field, and that even shielded regions receive approximately 25% of the dose to the unshielded areas